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Our Purpose Today 


• Overview of GSFC’s Supplier Assessment 
Program and its evolving approach 

• Share current analysis of assessment findings 
and associated trends 

• Address your questions! 
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What We Do and Why 


• Conduct independent assessments to assure the 
provision of quality products/services and foster 
improvements for NASA mission programs/projects 

- Assess conformance with contractual requirements and quality 
management standards (i.e., ISO 9001, AS9100) and the 
effectiveness of supporting processes 


Verify supplier corrective / preventive actions and improvements 

Share assessment results with GSFC mission projects, across 
NASA and other agencies 

Document improvements / benefits contributing to the fulfillment 
of requirements, mission success, and supplier capabilities 
(Success Stories) 
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What We Do and Why 


Supply Chain Management 


continued 


• Perform 45 to 50 assessments per year throughout the 
supply chains for GSFC mission projects 

- Primes and lower tiers: systems integrators/developers, and 
suppliers of subsystems, components and parts 

• Completed assessments of 135 unique suppliers since 
2007 

• Maintain information resource / database of supplier 
assessment results 

• Build teamwork and collaboration to foster value-added 
results 

Findings Actions Improvements 



Mission Success 
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Assessments / GSFC Mission Portfolio 


Assessments in FY 2012 supported GSFC’ s portfolio 
of mission programs / projects, including: 


• James Webb Space Telescope (JWST) 

• Mars Atmosphere and Volatile EvolutioN (MAVEN) Mission 

• Geostationary Operational Environment Satellite-R series 
(GOES-R) -- Flight and Ground projects 

• Global Precipitation Measurement (GPM) Mission 

• Landsat Data Continuity Mission (LDCM) 

• Magnetospheric MultiScale (MMS) Mission 

• Radiation Belt Storm Probes (RBSP) Mission 

• Tracking and Data Relay Satellites (TDRS) 

• Space Network Ground Segment Sustainment (SGSS) 

• Joint Polar Satellite System (JPSS) 
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Supplier Assessments 

FY 2012 

~ Locations 
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Analysis of Assessment Findings 
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Distribution of Assessed Suppliers 

09/2011 - 08/2012 


>250 Employees 
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Top Ten Findings by Category 



> 250 Employees 
76-250 Employees 
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Case Examples: Findings Over Time 


g Customer 

University ■ Observation 

□ Non-compliance 



2009 2011 2012 

Full Full Follow-up 


Small Co. = < 75 Employees 



2011 2012 
Full Follow-up 


Six findings from 2011 (35%) recurred in 
2012 

Reduction of findings over time shows 
improvements 

Calibration and GIDEP findings in all three 
assessments indicate recurring 
noncompliance to contractual requirements 
due to ...ineffective corrective actions, 
insufficient management commitment 


Five recurring findings in 2012 (38%) in 
contracts, document control, non- 
conforming material, industrial safety, 
and information security 
Reduction of findings over time shows 
improvements ... but sustained, 
systematic commitment is needed .... 
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Case Examples: Findings Over Time 


I Customer 

Medium Co. = 76-250 Employees ^ observatlon Large Co. = > 250 Employees 

Non-compliance 



2007 2011 2012 

Full Full Follow-up 


• Sixteen findings from 2011 (55%) 
remained unresolved in 2012 

• Evaluation of suppliers and ESD 
control were recurring problems in 
2007, 2011, and 2012. 

• Corrective Actions not effective in 



2007 2009 2010 2012 

Full Full Full Full 


• Growth in findings often indicates 
degradation of quality management 
system ... due to ? 


addressing findings ... overall lack of 
process control 



Safety and Mission Assurance Directorate, Goddard Space Flight Center 






*7 7 / pause fur a moment so you can let this information sink in . 




Supply Chain Management 

Illustrative Findings 


• No rational for accepting risk 

• Reliability data incorrect due to misuse of stress factors 

• Proof load testing requirements for fixtures/carts not complete 

• Peer reviews not performed by an independent person 

• Risk Management Plan does not fulfill NASA requirements 

• Information Security not addressed 

• Product not built to NASA Workmanship Standards as required by 
contract 

• Unidentified material used on delivered product 

• Conformal coating thickness measurement not performed 
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Illustrative Findings 


• Contractual requirements not known by suppler, e.g. calibration 
requirements 

• Calibration requirements not flowed to Calibration Service providers 

• Subcontractors not current with latest Mission Assurance 
Requirements (MAR) 

• Critical Data Requirements List (CDRL) items not approved as 
required 

• Purchasing does not require homogenous lots of fasteners as 
contractually required 

• Statement of Work (SOW) requirements not flowed to suppliers and 
sub-contractors 

• Product shipped although Statement of Work (SOW) in “Draft” state 


• Contract review process not performed or adequately documented by 
supplier 
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Assessment Approach 

Highlights 


• Suppliers systematically prioritized for assessments in 
coordination with projects / Chief Safety & Mission 
Assurance Officers 


• Assessments conducted by an expert team under GSFC 
Supply Chain Management leadership 

• Standard scope includes the supplier’s implementation 
and fulfillment of: 

• Quality Management Standards, as applicable (iso 9001, AS9100) 

• Contractual requirements for NASA projects 

• Internal Quality Management System and supporting processes 

• Assess the supplier’s conformance with requirements 
and the effectiveness of supporting processes 


Safety and Mission Assurance Directorate, Goddard Space Flight Center 




Supply Chain Management 

Assessment Approach 

Highlights - continued 


• Assessment Plan customized based on planning 

information collected from projects (concerns, risks) and 
the supplier (Assessment Planning Tool) along with past 
assessment / audit results 

• Risk-focused vs. waterfront 


• Methods include review and sampling of documents / 
records, interviews of management and personnel, 
direct observation of operations, and evaluation of key 
processes 


• Findings (nonconformances, observations and 
commendations) based on requirements, objective 
evidence, sound industry practice, and professional 
judgment 


• Nonconformance findings based on requirements 

• AS9100C used as the overall analytical framework 
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Assessment Approach 

Highlights - continued 

• Supplier strengths documented as positive comments 

• Supplier’s corrective / preventive actions requested for 
initial review within 30 days from the conclusion of the 
assessment visit 

• Follow-up assessment conducted for eyes-on” 
verification of the effective implementation of actions (via 
people, processes, tools/systems) to resolve findings 

• May also focus on new or related concerns as warranted 

• Identify and document Success Stories on resulting 
improvements / benefits contributing to fulfillment of 
requirements, mission success, and supplier capabilities 



Safety and Mission Assurance Directorate, Goddard Space Flight Center 



Supply Chain Management 

Assessment Approach 

Highlights - continued 

• Open communication with the supplier 

• Assessment Plan provided in advance for feedback / revision 

• Daily debriefs and Assessment Out-Brief 

• Questions / concerns addressed throughout the assessment 
process 

• Assessment Process Evaluation collected from the supplier 

• Assessment Report provided for Supplier comment prior to 
distribution 



Value-Added Results for NASA and the Supplier 
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Process Overview 

Supplier Assessments 


Plan Assessment 

• Selection Factors 
►Scheduling and Coordination 

• Assessment Plan 



/ 


Assessment Follow-up 

• Review Corrective / Preventive 
Actions 

• On-site Verification 
• Reporting 

•Document Success Stories 


i 


Conduct Assessment 

• GSFC-led Expert Team 
• Document Findings / Results 
• Reporting 



Knowledge Sharing 
• Assessment Database 
"Analytics 

• NASA Projects 
NASA Acquisition Processes 
•NASA Centers 
•Other Agencies 
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Assessment Team Staffing 


• Led by GSFC Supply Chain Management team member 


• Dave Campbell 

• Matt Fox 

• Charles Kim 

• Jonathan Root 

• Louis Thomas 



• Assessors from the NASA Audits, Assessments and 
Assurance (A3) Services contract 

• Assessors / observers from GSFC, other centers / 
agencies and prime contractors 
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Questions or Comments? 




A quality process addresses two separate, intertwined 

questions: 

Are we doing the right things? 

Are we doing things right? 

Townsend and Gebhardt 
Lessons in Leadership Participation and_ Measurement 
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Requirements 
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Quality Management Process Model 



Continual Improvement of the 
Quality Management System 
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Assessment Planning Tool 


Purpose 

The purpose of this document is to collect information for the planning of supplier assessments performed for the NASA 
Goddard Space Flight Center under the NASA Audits, Assessments, and Assurance (A3) Services contract. All 
information obtained is for U.S. Government purposes and is protected under provisions in NASA FAR Supplement 
Clause 1852.237-72 “ACCESS TO SENSITIVE INFORMATION.” 

General Business Information 

(1) Legal Business Name 

(2) CAGE Code 

(3) Doing Business As (DBA) Name 

(4) Website 

(5) Street Address 

(6) City 

(7) State or Province 

(8) Postal Code 

(9) Country 

(10) Business Phone 

Supplier Point of Contact 

(1 1 ) Primary Contact Name 

(12) Title 

(13) Telephone No. 

(14) E-mail 

(15) Alternate Contact Name 

(16) Title 

(17) Telephone No. 

(18) E-mail 

Facility / Number of Employees 

(19) Facility Total Area (Sq. Ft): 

Number of Buildings: 

Area Dedicated to Manufacturing (Sq. Ft.): 


(20) Total Number of Employees: 

Management / Administration: 
Engineering / Development: 
Manufacturing / Production: 
Quality Assurance / Safety 
Product / Customer Support 


(21) Clean rooms: 

Type (IS014644 Class): Area (Sq. Ft): 

Type (IS014644 Class): Area (Sq. Ft): 

Type (IS014644 Class): Area (Sq. Ft): 

Business Profile 

(22) 

List a description of the products and/or services that the business provides. Attach equipment list, capabilities 
list, brochure, or catalog if appropriate. 

(23) 

Does the facility use government furnished equipment or property? 

□ Yes 

□ No 

(24) 

How long has the business been at its current facility? 

Year(s) 

(25) 

Has the company changed ownership in the past year? 

□ Yes 

□ No 

(26) 

What percentage of the total business does NASA contracts currently represent? % 

(27) 

Is the business currently bidding on any NASA contracts? 

□ Yes 

□ No 

(28) 

In the past year, has the number of n stayed about the same n increased > 10% □ Decreased > 10% 

employees: 7 - - 
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Judith Bruner 
(Director) 

Richard Barney 
(Deputy Director) 

Eric Isaac 

(Associate Director) 

Jesse Leitner 
(SMA Chief Engineer) 


Directorate Resources 
Management Office 
(300.1) 

Curtis Johnson 

(Directorate Resources Manager) 


Joe Wonsever 

(Chief Technical Assessments Engineer) 
Robert Savage 

(Assistant Director for SMA @ Wallops) 
VACANT 

(Assistant to the Director) 
Jeanine Doherty 
(Administrative Officer) 


Information and Assurance Technology Services 
Sanjeev Sharma 
(IT Manager) 


EEE Parts/Workmanship Group 
Michael Sampson 

(NASA EEE Parts Assurance Group Program Manager) 
Jeannette Plante 

(Workmanship Program Manager) 


System Review 


Institutional Support 


Mission Support 


Software Assurance 


Occupational Safety & 

Office (301) 


Office (302) 


Division (320) 


Office (320.1) 


Health Division (350) 

Carolyn Dent 


Cindy Mead 


Mike Kelly 


Sue Sekira 


Patrick Hancock 

(Chief) 


(Chief) 


(Chief) 


(Office Chief) 


(Chief) 

VACANT 


Dave Campbell 


VACANT 


Daniel Simpson 

(Deputy Chief) 


(Deputy Chief) 


(Deputy Chief) 


(Deputy Chief) 


Supply Chain 
Management 
Team 


Systems Safety 


Reliability & Risk 


Branch (321) 


Analysis Branch 




(322) 


Bo Lewis 


Anthony Diventi 


(Branch Chief) 


(Branch Chief) 
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Mission Assurance 
Branch (323) 
Rob Sticka 
(Branch Chief) 
Vacant 

(Associate Branch 

Oi l ier) 
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Assessment Report Distribution 


• Assessment Report finalized after consideration of any supplier 
comments or suggested corrections 

• Identify strengths (positive comments, commendations) 

• Summarize findings, highlighting key areas for corrective action / 
improvement 


Distributed to the Supplier, NASA GSFC program / project offices 
and other NASA centers / U.S. government agencies upon 
request ; available for use in NASA procurement processes 
Supplier encouraged to share the report and corrective / 
preventive actions with its direct customers for NASA programs / 
projects 

• Reinforce supplier / customer communications 
Assessment report includes attached outbriefing, findings and 
attendance list 
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